An Introduction to Item Response Theory Using the Need for Cognition Scale

This paper provides an introduction to two commonly used item response theory (IRT) models (the two-parameter logistic model and the graded response model). Throughout the paper, the Need for Cognition Scale (NCS) is used to help illustrate different features of the IRT model. After introducing the IRT models, I explore the assumptions these models make as well as ways to assess the extent to which those assumptions are plausible. Next, I describe how adopting an IRT approach to measurement can change how one thinks about scoring, score precision, and scale construction. I briefly introduce the advanced topics of differential item functioning and computerized adaptive testing before concluding with a summary of what was learned about IRT generally, and the NCS specifically. Many of the constructs psychologists are interested in studying are not directly observable. Examples include depression, intelligence, extroversion, and need for cognition. To study these constructs, researchers will often assemble a set of items which are believed to be related to the construct of interest. This set of items (also known as a scale, or questionnaire) can be viewed as one possible operationalization of the construct of interest. To the extent that responses to the items are related to the construct we intend to assess, the observed item responses can provide information about the unobserved construct. For this to be possible, we must have some model (often called a measurement model) to relate the observed response to the underlying construct. There are many measurement models which could serve as a link between the observed and latent worlds. One long standing measurement model is true score theory, which is part of classical test theory (CTT). CTT is perhaps most strongly associated with reliability coefficients such as coefficient alpha (Cronbach, 1951) and has a long history in psychology. As useful as CTT can be, recent advances in latent variable modeling have provided access to measurement models which are more flexible and more powerful than CTT.